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Abstract 

The  US  Army  Test  and  Evaluation  Command  (ATEC) 
currently  is  responsible  for  providing  operational 
meteorological  support  to  research,  development,  test, 
and  evaluation  (RDT&E)  activities  at  eight  Army 
installations.  The  Four-Dimensional  Weather  (4DWX) 
meteorological  support  system  used  to  provide  that 
support  was  developed  by  the  National  Center  for 
Atmospheric  Research  (NCAR)  in  collaboration  with 
ATEC  meteorologists.  A high-resolution  (mesoscale) 
numerical  weather  prediction  (NWP)  model  is  a major 
4DWX  component  in  operational  use  at  seven  ATEC  test 
centers.  This  paper  describes  current  4DWX  NWP 
capabilities  and  discusses  how  the  new  Department  of 
Defense  (DoD)  Dedicated  High  Performance  Computing 
Project  Investment  (DHPI)  at  Dugway  Proving  Ground 
(DPG)  will  enhance  NWP  forecast  support  to  ATEC  field 
and  virtual  tests. 

1.  Current  4DWX  NWP  Capabilities 
1.1.  The  4DWX  Modeling  System 

The  4DWX  system  provides  information  about 
meteorological  processes  in  both  the  three  space 
dimensions  and  time.  The  4DWX  NWP  modeling  system 
has  two  closely  related  components:  one  assimilates 

meteorological  data  and  the  other  produces  the  forecasts. 
Both  components  are  based  on  the  same  meteorological 
model — the  Pennsylvania  State  University  (PSU)/NCAR 
mesoscale  model  version  5 (MM5,  Grell,  et  al.,  1995), 
although  the  transition  to  the  next-generation  community 
Weather  Research  and  Forecast  (WRF)  model  is 
underway.  The  4DWX  real-time  four-dimensional  data 
assimilation  (RT-FDDA)  engine  ingests  meteorological 
data  as  they  become  available,  producing  model- 
assimilated  data  sets  that  both  define  the  current 
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conditions  of  the  atmosphere  and  serve  as  initial 
conditions  for  the  model  forecasts.  The  other  4DWX 
NWP  component  produces  forecasts  of  up  to  48-h 
duration,  with  the  forecast  length  depending  on  the 
specific  need  and  the  computer  resources  available. 

Data  assimilation  is  done  by  Newtonian  relaxation, 
which  involves  adding  nudging  terms  to  the  model 
predictive  equations.  These  terms  force  the  model 
solution  at  each  grid  point  to  observations  in  proportion  to 
the  difference  between  the  model  solution  and  the 
observations.  This  approach  was  chosen  because  it  is 
robust,  relatively  efficient  computationally,  allows  the 
model  to  ingest  data  continuously  rather  than 
intermittently,  and  the  full  model  dynamics  are  part  of  the 
assimilation  system,  which  means  that  analyses  contain 
all  locally-forced  small-scale  features.  Additionally,  the 
nudging  methodology  does  not  unduly  complicate  the 
structure  of  the  model  code. 

1.2.  Example  Applications  of  the  4DWX 
Modeling  System 

Examples  are  provided  below  of  the  use  of  the 
current  4DWX  NWP  system  for  various  applications  that 
require  HPC  resources.  The  first  compares  warm-season 
rainfall  predictions  by  the  White  Sands  Missile  Range 
(WSMR)  4DWX  system  with  (1)  observations,  and  (2) 
forecasts  by  coarser-resolution  operational  models  of  the 
National  Weather  Service  (NWS).  The  second  example  is 
application  of  the  4DWX-system  to  define  the  fine-scale 
climate  over  regions  where  observations  are  not  available 
because  of  small  data  voids  between  observations  or  data 
denial  over  large  areas  of  the  world. 

1.2.1.  Warm-Season  Rainfall  in  the  Southwest  US 

Precipitation  can  have  important  impacts  on  test  and 
other  operations  at  any  of  the  ATEC  ranges.  The  White 
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Sands  Missile  Range  (WSMR)  4DWX  forecast  system 
was  chosen  for  evaluation  of  the  performance  of  the 
4DWX  system  in  predicting  precipitation  because  of  the 
prevalence  of  summertime  convection  from  the  North 
American  monsoon,  the  importance  of  predicting  the 
lightning  hazards  associated  with  this  convection,  and  the 
proximity  of  the  WSR-88D  Doppler  weather  radar  at 
Holloman  Air  Force  Base  for  use  in  the  evaluation. 
Additionally,  the  fact  that  the  precipitation  is  related  to 
the  combined  effects  of  the  large-scale  North  American 
monsoon  and  modulation  by  the  local  orography  provided 
an  opportunity  to  evaluate  the  model’s  ability  to  correctly 
render  both  large-scale  and  small-scale  processes. 

We  compared  the  WSMR  4DWX  precipitation 
forecasts  for  August  2005  with  those  by  the  NWS’s  North 
American  Model  (NAM)  and  Rapid  Update  Cycle  (RUC) 
model.  The  WSMR  4DWX  model  has  three  nested 
computational  grids,  and  the  one  that  we  used  for  model 
evaluation  was  the  middle  one,  which  has  a 10-km 
horizontal  grid  increment.  This  resolution  is  similar  to 
those  of  the  NAM  and  RUC,  which  have  horizontal  grid 
increments  of  12  and  13  km,  respectively.  Evaluation  of 
convective  precipitation  forecasts  is  well  known  to  be 
especially  problematic  because  it  is  difficult  to  measure 
the  large  spatial  variations  in  convective  precipitation. 
We  compared  the  precipitation  forecasts  with  the  National 
Centers  for  Environmental  Prediction  (NCEP)  Stage-IV 
analyses,  which  are  constructed  by  compositing  WSR- 
88D  radar  and  rain  gauge  estimates.  Only  a subjective 
evaluation  is  reported  on  here.  The  Stage-IV  analyses  in 
the  Southwest  US  tend  to  underestimate  the  actual 
precipitation  amounts  because  the  higher  terrain  blocks 
the  radar  "beam  and  there  is  a paucity  of  rain  gauges  in 
some  areas.  The  forecasts  were  evaluated  for  the  entire 
month  of  August  2005,  a period  during  which  the  North 
American  monsoon  was  active  in  the  Southwest  US. 
With  eight  4DWX  forecast  cycles  per  day,  the  evaluation 
was  based  on  over  200  forecasts,  most  of  which  had  some 
rainfall. 

Figure  la  shows  the  observed  (Stage-IV  analysis) 
monthly  mean  precipitation  accumulation  for  the  3-h 
period  from  2100-0000  UTC  (1400-1700  Local  Time 
[LT]).  The  higher  precipitation  amounts  generally  are 
restricted  to  areas  of  terrain  elevation  greater  than  2,000 
m above  mean  sea  level  (MSL),  the  “altitude  oases”  that 
are  well  known  in  arid  areas.  For  example,  the  north- 
south-oriented  area  of  higher  precipitation  in  eastern  New 
Mexico  is  related  to  the  Sacramento  Mountains,  while  the 
northwest-southeast-oriented  feature  in  eastern  Arizona 
and  western  New  Mexico  coincides  with  the  Mogollon 
Plateau  in  Arizona  and  its  extension  into  New  Mexico. 
For  this  3-h  time  period,  there  was  at  least  some  moist 
convection  over  this  area  on  26  days  of  the  month. 
Figures  lb-d  show  the  monthly  mean  precipitation 
forecasted  for  this  3-h  period  by  the  NAM,  RUC,  and 


WSMR  4DWX  models.  All  three  models  capture  the 
observed  dryness  in  the  southwest  part  of  the  area  and  the 
observed  north-south  oriented  strip  of  dryness  in  the  Rio 
Grande  Valley  in  west-central  New  Mexico.  Recalling 
that  the  observational  data  are  expected  to  under-estimate 
precipitation,  the  WSMR  4DWX  forecasts  appear  to 
better  capture  the  correct  spatial  scales  and  amplitudes  of 
the  rainfall  over  most  of  the  area. 

1.2.2.  Regional-Climate  Downscaling 

The  4DWX  NWP  model  also  can  be  used  to  provide 
high-resolution  depictions  of  regional  climates.  Estimates 
of  the  fine-scale  climate  of  a region  can  be  useful  for  long 
range  ATEC  test  planning,  especially  in  data  sparse 
regions.  As  one  test  of  this  capability,  we  considered  the 
eastern  Mediterranean  and  the  adjacent  countries  of  the 
Middle  East,  where  the  Mediterranean  Sea  is  a large  data 
void.  Our  objective  was  to  use  satellite  estimates  of 
rainfall  to  confirm  that  the  model  was  reasonably 
replicating  the  small-scale  aspects  of  the  precipitation 
climate.  We  choose  precipitation  for  this  test  because  it  is 
sensitive  to  many  other  variables  and  is  an  excellent 
single  indicator  that  most  other  physical  variables  are 
being  simulated  correctly. 

We  ran  the  4DWX  NWP  model  in  the  analysis  mode 
for  six  Januaries  (2001  through  2006)  using  global 
analyses  archived  by  the  NCAR-NCEP  Reanalysis  Project 
for  lateral-boundary  conditions.  Surface  and  upper-air 
observations  were  assimilated  on  the  model  grids  to 
provide  the  best  possible  model  climatology  of  the  region. 
The  left  panel  of  Figure  2 shows  the  average  January 
precipitation  amount  predicted  by  the  4DWX  model 
simulations.  Significant  precipitation  maxima  over  land 
are  seen  along  the  coastlines  of  the  Levant,  Turkey,  and 
Greece.  There  also  is  significant  precipitation  over  the 
Mediterranean  to  the  south  of  Greece.  The  precipitation 
gradient  is  very  large  along  the  North  African  coastline. 
The  gradient  is  somewhat  less  along  the  Levant  and 
Turkey  coasts,  but  it  is  still  large.  The  satellite 
precipitation  estimates  in  the  right  panel  of  Figure  2 are 
consistent  with  the  model  estimates,  showing  the  greatest 
amounts  along  the  coasts  of  the  Levant,  Turkey,  and 
Greece,  with  large  coastal  gradients.  However,  the  model 
predicted  less  precipitation  over  the  central  and  southern 
Mediterranean  than  was  derived  from  satellite  imagery. 

The  model-defined  fine-scale  climate  of  the  weather 
variables  also  can  be  used  as  input  to  coupled  models  to 
produce  specialized  climatologies.  For  example,  Figure  3 
illustrates  the  use  of  fine-scale  climatology  to  assess  the 
consequences  of  the  release  of  hazardous  material  from  a 
location  in  East  Asia  during  the  month  of  May.  The 
model  analysis  for  each  day  in  the  period  of  record 
included  boundary  layer  winds  and  stability  (heat  fluxes). 
We  used  a DoD  transport  and  diffusion  model  (Sykes,  et 


256 


al.,  2007)  to  calculate  dosage  (time-integrated 
concentration)  resulting  from  the  hypothetical  release  of 
hazardous  material  at  a particular  location.  We  then 
aggregated  this  ensemble  of  plumes  into  an  analysis  of  the 
probability  of  the  surface  dosage  exceeding  a specified 
threshold.  As  shown  in  Figure  3,  there  are  preferred 
transport  corridors  to  the  northeast  and  southeast  of  the 
hypothetical  source.  Higher-resolution  plume  displays 
illustrate  fine  scale  orographic  and  coastal  influences  on 
the  winds,  which  could  only  be  defined  with  a model  such 
as  the  ATEC  4DWX  system. 

2.  Ensemble  Prediction 

All  weather  predictions  are  subject  to  uncertainty 
arising  from  imperfections  in  both  the  initial  conditions 
for  a forecast  and  in  the  model  that  propagates  those 
initial  conditions  into  the  future.  Initial  condition  (IC) 
uncertainty  is  inevitable  because  the  discrete  nature  of 
atmospheric  observations  and  the  uncertainties  in  those 
observations  prevent  a perfect  estimate  of  the  state  of  the 
atmosphere  at  any  time.  Model  uncertainty  is  equally 
unavoidable  because  the  system  of  equations  that 
describes  atmospheric  evolution  requires  numerical  and 
physical  approximations.  The  objective  of  the  DPG 
DHPI  is  to  enhance  4DWX  NWP  capabilities  by 
developing  an  operational  mesoscale  (ensemble-based) 
probabilistic  forecast  system  to  quantify  the  uncertainty  in 
4DWX  forecasts  for  major  ATEC  field  tests  wherever 
they  occur. 

The  4DWX  probabilistic  forecast  approach  is  to 
replace  a single  NWP  run  with  an  ensemble  of  model 
runs,  with  the  initial  conditions  and  model  physics  of  the 
ensemble  members  varied  to  bound  the  range  of 
uncertainty.  Major  challenges  include  sampling  error 
resulting  from  an  inability  to  account  for  the  number  of 
degrees  of  freedom  in  the  atmosphere,  and  also 
fundamental  deficiencies  in  models  that  limit  the 
representation  of  uncertainty  in  the  prediction  process. 
The  4DWX  ensemble  forecast  is  an  attempt  to  account  for 
uncertainties  in  both  initial  conditions  and  model  physics 
parameterizations,  and  to  ensure  that  the  probabilistic 
forecasts  adhere  to  the  probability  distribution  of 
observations. 

Initial-condition  uncertainty  is  best  handled  with 
methods  that  consider  both  observational  uncertainty  and 
uncertainty  in  the  forecast  process.  Currently,  the  best 
available  methods  are  of  a class  of  linear  Monte-Carlo 
filters  related  to  the  Kalman  filter.  The  ensemble 
transform  Kalman  filter  (ETKF;  e.g.,  Wang  and  Bishop, 
2003),  which  is  attractive  because  an  ensemble  of  initial 
conditions  can  be  obtained  efficiently,  is  a Monte-Carlo 
sample  of  the  spatially  covarying  distribution  of  error  in 
the  best  estimate  of  initial  conditions.  Although  it  is  an 


approximation  to  the  optimal  linear  Kalman  filter,  the 
dynamics  of  the  model  allow  nonlinear  temporal 
propagation  of  the  uncertainty  from  one  forecast  cycle  to 
the  next.  Because  a computationally  feasible  ensemble 
cannot  represent  all  of  the  degrees  of  freedom  in  a 
mesoscale  prediction,  sampling  error  can  accumulate  in 
any  ensemble  filter  that  runs  over  several  forecast  cycles. 
The  new  DoD  high  performance  computing  (HPC) 
platform  being  employed  for  this  work  enables 
development  of  an  ETKF  system  with  more  ensemble 
members  than  are  typically  used,  which  should  help  to 
mitigate  the  sampling  error  problem. 

Model  physics  uncertainty  can  be  handled  by 
including  a distribution  of  models  in  an  ensemble,  where 
the  distribution  is  populated  by  switching  model  options 
for  subgrid  scale  physical  parameterizations  or  by  adding 
entirely  different  modeling  systems.  The  implicit 
assumption  is  that,  for  any  given  forecast  scenario,  we 
cannot  know  a priori  which  model  will  perform  the  best 
according  to  a given  metric,  and  that  all  of  them  are 
equally  probable.  While  this  assumption  is  not 
necessarily  valid,  the  multi-model  approach  has  proved  to 
add  potential  value  to  a probabilistic  forecast.  Combined 
with  the  ETKF,  the  multi-model  approach  augments  the 
initial-condition  uncertainty  with  model  uncertainty.  The 
new  HPC  allows  a large  number  of  model  configurations 
to  be  run  operationally,  and  techniques  such  as  self- 
organizing maps  are  being  adopted  to  objectively  remove 
model  configurations  that  systematically  perform  poorly. 

Finally,  a large  ensemble  of  model  runs  still  cannot 
produce  a probabilistic  forecast  that  exactly  matches 
atmospheric  probabilities.  Consequently,  post-processing 
calibration  is  needed  to  provide  a prediction  with 
probabilities  that  agree  with  observed  probabilities  of 
weather  events.  Basic  statistical  techniques  such  as 
quantile  regression  show  great  promise  in  calibration,  but 
huge  samples  of  forecasts  are  needed.  Accumulation  of 
massive  datasets  of  daily  ensemble  forecasts  for  a long 
record  (a  year  or  more)  is  necessary  to  tune  the 
probabilistic  forecasts.  Daily  minimum  and  maximum 
surface  temperatures,  which  can  be  important  for 
personnel  safety  and  equipment  performance  at  ATEC 
test  ranges,  are  the  first  forecast  parameters  that  will  be 
subject  to  calibration  in  the  4DWX  system. 

The  initial  4DWX  ensemble  forecast  implementation 
consists  of  55  members  on  the  three  computational 
domains  (30,  10,  and  3.3  km  horizontal  grid  spacing)  with 
a cycling  ETKF  and  multiple  models.  Forecasts  are 
launched  from  ETKF  analyses  every  6-h,  and  provide 
24-h  or  greater  forecasts  for  users.  Lateral  boundary 
conditions  are  obtained  from  the  same  sources  as  the 
deterministic  (single)  4DWX  NWP  model  runs. 
Probabilistic  evaluation  is  ongoing,  and  calibration  of 
daily  minimum  and  maximum  temperature  forecast 
distributions  with  logistic  regression  is  beginning. 
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Figure  1.  For  August  2005,  the  monthly  mean 
precipitation  accumulation  for  the  3-h  period  from 
2100-0000  UTC  (1400-1700  LT)  based  on  (a)  the  NCEP 
Stage-IV  analysis,  (b)  the  NAM  model  (12km  grid 
increment),  (c)  the  RUC  model  (13-km  grid  increment) 
and  (d)  the  WSMR  4DWX  model  (10-km  grid 
increment) 
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Figure  2.  January  precipitation  for  2001-2006  based 
on  4DWX-model  simulations  (left)  and  data  from  the 
merged  Tropical  Rainfall  Measurement  Mission 
(TRMM)  satellite  and  rain  gauges  (right) 
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Figure  3.  The  gray  shades  indicate  the  probability  (%) 
that  the  dosage  from  a release  of  hazardous  material 
at  0700  local  time  in  May  in  Korea  will  exceed  a 
particular  threshold 
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